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ABSTRACT 

Objective: Randomised controlled trials (RCTs) are 
often considered as the gold standard for assessing 
new health interventions. Patients are randomly 
assigned to receive an intervention or control. The 
effect of the intervention can be estimated by 
comparing outcomes between groups, whose 
prognostic factors are expected to balance by 
randomisation. However, patients' non-compliance with 
their assigned treatment will undermine randomisation 
and potentially bias the estimate of treatment effect. 
Through simulation, we aim to compare common 
approaches in analysing non-compliant data under 
different non-compliant scenarios. 
Settings: Based on a real study, we simulated 
hypothetical trials by varying three non-compliant 
factors: the type, randomness and degree of non- 
compliance. We compared the intention-to-treat (ITT), 
as-treated (AT), per-protocol (PP), instrumental 
variable (IV) and complier average casual effect (CACE) 
analyses to estimate large (50% improvement over the 
control), moderate (25% improvement) and null (same 
as the control) treatment effects. Different approaches 
were compared by the bias of estimate, mean square 
error (MSE) and 95% coverage of the true value. 
Results: For a large or moderate treatment effect, the 
ITT estimate was considerably biased in all scenarios. 
The AT, PP, IV and CACE estimates were unbiased 
when non-compliant behaviours were random. The IV 
estimate was unbiased when non-compliant behaviours 
were symmetrically dependent on patients' conditions. 
The PP estimate was mostly unbiased when patients in 
the control group did not have access to the 
intervention. When the intervention was not different 
from the control, the ITT was less biased than the 
other approaches. Similar results were found when 
comparing the MSE and 95% coverage. 
Conclusions: The standard ITT analysis under non- 
compliance is biased when the intervention has a 
moderate or large effect. Alternative analyses can 
provide unbiased or less biased estimates. Based on 
the results, we make some suggestions on choosing 
optimal approaches for analysing specific non- 
compliant scenarios. 



Strengths and limitations of this study 



We compared different methods to analyse non 
compliant data by simulating hypothetical rando 
mised controlled trials. 

Different non-compliant scenarios were gener- 
ated by three factors: the type, randomness and 
degree of non-compliance. 
The simulation framework and parameters were 
built on a real study. 
Patients' prognostic factors and missing data 
due to withdrawal were not considered in the 
simulation. 



Correspondence to 

Dr Lehana Thabane; 
thabanl@mcmaster.ca 



BACKGROUND 

Randomised controlled trials (RCTs) are 
often considered as the gold standard for 



assessing new health interventions where 
patients are randomly assigned to receive an 
intervention or control (eg, placebo). Since 
patients' prognostic factors are expected to 
balance by randomisation patients' outcomes 
can be directly compared between groups to 
infer the effect of a treatment. In many 
cases, patients may not fully comply with 
their assigned treatment according to the 
protocol. Such protocol violation compro- 
mises the 'fair' comparison, which is pro- 
tected by randomisation, and will potentially 
bias the estimate of treatment effect. 
Analysing RCTs subject to non-compliance 
can be challenging. While different analyses 
have been proposed to deal with non- 
compliance, the bias of treatment effect esti- 
mate is rarely compared among different 
approaches. Result interpretations also vary 
depending on the nature of non-compliance 
and the objective of a trial. Some RCTs, 
known as pragmatic trials, 1-3 are primarily 
designed to guide clinical practice. Their 
goal is often to assess whether an interven- 
tion will work in routine practice. In contrast, 
non-pragmatic trials usually focus on the bio- 
logical efficacy of an intervention. Despite 
the objective, an analysis that provides an 
unbiased or less biased estimate of treatment 
effect is always desirable. In this study, we 
compare common approaches to analyse 
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non-compliant data in RCTs. The results will provide 
useful knowledge in choosing optimal methods for dif- 
ferent non-compliant scenarios. 

This study was motivated by an RCT that compared 
the integrated care organised through the Children's 
Treatment Network (CTN) with the usual care directed 
by parents for managing children with special health- 
care needs. 4 The CTN coordinated community 
resources to deliver comprehensive health services for 
the target children and their families. The hypothesis 
was that the target children's health outcomes would 
improve by receiving integrated, proactive and necessary 
services tailored for them. While the use of RCTs in 
assessing CTN-like interventions has been promising, 
non-compliant rates are generally high in these trials. 
This is largely due to the complexity of implementing 
multidisciplinary interventions in real-life settings. 

Intention to treat (ITT), as treated (AT) and per proto- 
col (PP) are common approaches to analyse non- 
compliant data in RCTs. The ITT analysis is considered as 
the gold standard 5 but can be problematic for some scen- 
arios. 6 Multiple methods are often recommended for 
analysing RCTs with substantial non-compliance. 7-9 A lit- 
erature review 10 randomly selected 100 RCTs published 
in high impact journals in 2008. Of 98 RCTs which 
reported non-compliance, 46 employed variations of PP 
analyses in addition to an ITT analysis. Another class of 
methods to deal with non-compliance includes instru- 
mental variable (IV) and complier average causal effect 
(CACE) approaches. 11-15 A conceptual difference among 
all these methods is that the ITT, IV and CACE 
approaches estimate treatment effects by preserving ran- 
domisation or accounting for potential confounding, but 
the AT and PP approaches do not. There are also other 
proposed approaches to correct for non-compliance in 
RCTs, 16 for example, G-estimation and inverse probability 
weighted estimators. However, these methods have not 
been widely adopted. Therefore, we only included the 
ITT, AT, PP, IV and CACE methods in our comparison. 

Very few studies have compared these five methods on 
the bias of estimating treatment effects. Bang and 
Davis 17 had compared ITT, AT, PP and IV methods. 
They showed that ITT and IV analyses were biased in 
certain non-compliant cases. However, the authors did 
not include CACE analysis in their comparison and did 
not consider the situation where there was no crossover 
between treatment groups. This scenario is common 
when a new intervention is only accessible to patients 
who are offered it. In another study, McNamee 18 com- 
pared ITT, AT, PP and IV analyses and concluded that 
an ITT analysis was not always biased towards the null 
while AT and PP analyses were generally biased. Sheng 
and Kim 19 investigated the effect of non-compliance on 
ITT analysis of equivalence trials and showed that non- 
compliance did not always favour the null hypothesis, 
that is, no difference between treatment groups. 
Hertogh et af° concluded that the IV method could give 
insight into confounding by non-compliance in RCTs. 



Most of the previous studies did not consider different 
associations of non-compliant behaviours with patients' 
conditions. For example, patients with certain character- 
istics may always reject a new intervention. Also, there 
are partial non-compliant cases where patients receive 
parts of the intervention even if they did not fully 
comply with the protocol. In our simulation study, we 
considered additional non-compliant scenarios that were 
not considered by previous studies and compared the 
five common methods by the bias of estimate, the mean 
square error (MSE) and 95% coverage of the true value. 
Our objectives were to compare the performance of dif- 
ferent approaches in analysing non-compliant RCT data 
and make recommendations on optimal approaches 
under specific scenarios. 

METHODS 

Simulation framework 

In the CTN trial, over 50% of the children randomised 
in the CTN group did not fully comply with the inter- 
vention for various reasons. Primary and sensitivity ana- 
lyses showed that the effect of the CTN was not 
significant but the estimates varied in direction, magni- 
tude and precision. 4 This observation prompted us to 
further investigate the impact of non-compliance on esti- 
mating treatment effects. 

On the basis of the CTN setting, we simulated hypothet- 
ical RCTs where patients were randomly assigned to the 
intervention or usual care by a 1:1 allocation ratio. The 
parameters for generating hypothetical patients were esti- 
mated from the CTN trial. We simulated different non- 
compliant scenarios by varying three factors: (1) the type 
of non-compliers, (2) the randomness of non-compliance 
and (3) the degree of non-compliance. Our simulation 
framework is shown in figure 1. The design, conduct and 
reporting of this study has followed the guideline of 
designing and reporting simulation studies. 21 

Type of non-compliers 

We considered two types of non-compliers which were 
defined as: never-takers and always-takers. 22 Never-takers 
are patients who will always reject a new intervention if 
they are offered it. Always-takers will always receive a new 
intervention even if they are not offered it. Two scenarios 
were considered. In one scenario, we assumed that non- 
compliers were either never-takers or always-takers, which 
mimicked the situation where patients were able to get the 
intervention elsewhere even if they were not offered it. In 
the other scenario, we assumed that non-compliers were 
only never-takers, which mimicked the situation where the 
intervention was only accessible to patients who were 
offered it. In addition, we assumed that the intervention 
and usual care were the only treatment options. 

Randomness of non-compliance 

Non-compliant behaviours could be random or depend- 
ent on patients' conditions. In particular, we considered 
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Figure 1 The simulation 
framework. 



Non-compliant scenarios 



- Both always- and never-takers - 



- Random non-compliance - 



Dependent non- 
compliance 



L Only never-takers 



- Random non-compliance - 



_| Dependent non- |_ 
compliance 



Partial non-compliance 



Partial non-compliance 



Partial non-compliance 



Partial non-compliance 



six scenarios of dependent non-compliant behaviours 
that were studied by McNamee 18 : 

A. Patients with good conditions would always get the 
intervention while patients with poor conditions 
would always reject it; 

B. Patients with good conditions would always get the 
intervention; 

C. Patients with poor conditions would always reject the 
intervention; 

D. Patients with good conditions would always reject the 
intervention while patients with poor conditions 
would always get it; 

E. Patients with good conditions would always reject the 
intervention; 

F. Patients with poor conditions would always get the 
intervention. 

Patients' conditions were considered to be positively 
associated with their outcomes under usual care. We 
assumed that good condition represented an outcome 
score of at least 0.5 SDs above the group mean under 
usual care (assuming that a high score was a better 
outcome). Poor condition represented an outcome 
score of at least 0.5 SDs below the group mean under 
usual care. When there were no always-takers, only scen- 
arios C and E were considered. 

Degree of non-compliance 

Degree of non-compliance referred to the proportion of 
interventional components that a patient did not receive 
according to the protocol. The simplest case was 
all-or-none where compilers received 100% of the inter- 
vention and non-compliers received none of it. For 
multifaceted interventions, patients were likely to receive 
some components of the intervention even if they did 
not fully comply with the protocol. In addition, patients 
might only receive parts of the intervention because of 
the intervention fidelity. For example, a systematic 
review showed that many interventions of integrated 
care did not actually deliver all services as planned due 
to complexity of implementation. 23 Conversely, we also 
use the term 'degree of compliance' for the proportion 
of interventional components that a patient received 
according to the protocol. Non-compliance and compli- 
ance are used throughout this paper. 



All-or-none and partial non-compliance were consid- 
ered in our simulation. For all-or-none case, we consid- 
ered two compliance levels: all components (d=l) or 
none (d=0) of the intervention. A study reported that 
non-compliance rate could be as high as 30-40% for a 
treated population. 24 Therefore, we randomly selected 
30% of patients to receive a treatment opposite to what 
they were assigned for. For partial non-compliance, we 
considered four compliance levels: none (d=0), 
one-third (d=l/3), two-thirds (d=2/3) or all (d=l) com- 
ponents of the intervention. These four levels have been 
studied in a previous simulation study. 17 

Simulation procedures 

We employed a modified simulation model from the 
previous study. 17 Let Y x and Y 0 be a pair of counterfac- 
tual outcomes for a patient if he or she were in the inter- 
vention and the usual care groups, respectively. In 
practice, we can only observe one of the counterfactual 
outcomes because we can never observe both outcomes 
for any patient at the same time. Thus, causal inference 
is often made at population level instead of patient level. 
By adopting a marginal view, we define the causal effect 
for the treatment of interest (8) as 

8 = Ha - M<o> 

where fix and u 0 were the means of Y x and Y 0 , 
respectively. 

We chose u 0 =59 to be the effect of usual care. The 
effect of usual care was estimated from the CTN trial 
with an SD of 10. For the effect of treatment (ui), we 
chose three different cases: Ui=89 for a 50% improve- 
ment over the usual care; U!=74 for a 25% improvement 
over the usual care and ui=59 for no difference from 
the usual care. Each case was simulated separately. We 
then generated individual patient's counterfactual out- 
comes through a normal distribution: 

Y k ~ Normal(fx k , 10 2 ) and k = 0, 1. 

Thus, good condition was defined for a patient with 
Y 0 >64 (half SDs above the group mean under usual 
care) and poor condition was defined for a patient with 
Y 0 <54 (half SDs below the group mean under usual 
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care). A group indicator Z (1 intervention and 0=usual 
care) was generated for each patient from a Bernoulli 
distribution with equal probability of 0.5 of being 
assigned to either group. The observed outcome for a 
patient was calculated by 

y. = diYj + (1 - di)Y 0 , 

where di was the degree of treatment compliance with 
the protocol for patient i. For all-or-none case, d was 
either 1 or 0. For partial compliance, d took a value of 
0, 1/3, 2/3 or 1. 

In the CTN trial, 450 patients were needed to detect a 
minimum clinically important difference (MCID) of 15 
with 80% statistical power and 5% a. Using the same 
MCID, we chose to simulate 500 participants in each 
hypothetical trial. We estimated the SE of the treatment 
effect to be 1.53 from the CTN trial. Based on this esti- 
mate, at least 816 simulations were needed to produce 
an effect estimate within 1% accuracy of the MCID by 
the standard formula. 21 To have sufficient power, we 
chose to generate 1000 simulations per scenario. The 
steps of simulation are shown in figure 2. 

Statistical analysis 

This section describes the different methods that we 
compared. The estimated treatment effect was expressed 
as the difference of the mean score between groups. 
The methods were compared by the bias, MSE and 95% 
coverage. 21 Bias is defined as 

Bias = 8 — 8, 

where 8 is the average estimate of interest over all itera- 
tions and 8 is the true value. MSE is calculated by 

MSE = (8-8) 2 + (SE(8)) 2 , 

where SE(8) is the SD of the empirical distribution of 
the estimates from all iterations. Out of all iterations, 
95% coverage is the number of times the 95% CIs 
include 8. CI is calculated by normal approximation. 

Intention to treat 

In the ITT approach, patients are analysed by how they 
were randomised regardless of their actual compliance 
with treatment. The treatment effect was estimated by 



ditt 



7i ~ Yo 



where y 1 and y 0 were the mean outcome scores of the 
intervention and usual care groups, respectively. 

As treated 

The AT approach compares patients by the treatment 
they actually received. The treatment effect was esti- 
mated by 



Step 1: generate Y 1; Y 0 , and Z through the 
specified parameters and distribution. 



Step 2: degree of non-compliance 







Partial non-compliance 



Randomly select 30% 
patients, then assign d 
= 1-1 



Randomly assign d to a different proportion 
of patients. In the intervention group: 

50% for d= 1; 

10% for d =2/3; 

10% for d = 1/3; 

30% for d = 0; 

In the usual care group: 

50% for d = 0; 

30% for d = 1/3; 

10% for d = 2/3; 

10% for d = 1. 



Step 3: randomness of non-compliance 



If random 



Assign d=l if Y 0 > 64 
and d=0 if Y n < 54. 



Assign d=l if Y 0 > 64. 



If Scenario C 



Assign d=0 ifY 0 <54. 



Assign d=0 if Y 0 > 64 
and d=l if Y n < 54. 



Assign d=0 if Y 0 > 64. 



If Scenario F 



Assign d=l if Y 0 < 54. 



Step 4: type of non-compliers 



Only never-takers 



Both always- and never-takers 



Assign d = 0 if Z = 0 



Final step: compute 
y = d*Y 1 + (l-d)*Y 0 . 



e 



AT Ytreated yuntreated ' 



Figure 2 Summary of the simulation steps. Y-i=the 
counterf actual outcome for a patient in the intervention group. 
Y 0 =the counterfactual outcome for a patient in the usual care 
group. Z=randomisation indicator. d=the degree of compliance 
with the intervention. y=the simulated outcome for a patient. 
Scenario A: patients with good conditions will always get the 
intervention and those with poor conditions will always reject 
it. Scenario B: patients with good conditions will always get 
the intervention. Scenario C: patients with poor conditions will 
always reject the intervention. Scenario D: patients with good 
conditions will always reject the intervention and those with 
poor conditions will always get it. Scenario E: patients with 
good conditions will always reject the intervention. Scenario F: 
patients with poor conditions will always get the intervention. 

For patients with partial compliance, the treatment 
effect was estimated by regressing the degree of compli- 
ance (d) on the outcome in a linear regression model. 
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Also, a different notation, 0, was used to differentiate 
from the estimators of making causal inferences. 

Per protocol 

The PP approach excludes patients who did not fully 
comply with treatment protocol. The treatment effect 
was estimated by 

^PP yobserved compilers in the intervention 
yobserved compilers in the control . 

Instrumental variable 

The IV approach employs the randomisation indicator 
(Z) as an IV to adjust for the proportion of non- 
compliant patients. The theory and assumptions of IV 
analysis have been thoroughly discussed in the litera- 
ture. 12 13 We used the standard IV estimator for linear 
models: 

di - d 0 

The Fieller's theorem was used to calculate the SE of 
the estimate. 17 

Compiler average causal effect 

The CACE method estimates the treatment effect 
among compilers. The assumptions and the casual 
framework of CACE have been discussed elsewhere. 14 15 25 
The treatment effect was estimated by 

Scace = E[y : - y 0 1 compilers]. 

There are two general approaches to the CACE infer- 
ence 25 : the maximum likelihood approach by expectation- 
maximisation (EM) algorithm and the Bayesian approach. 
The CACE analysis in this paper was conducted in Mplus 
(V7; Mac OS X 10.6.8) Los Angeles, GA, Muthen & 
Muthen, which employed the EM algorithm. The rest of 
the analyses and simulations were performed in R V2.15.2. 

Cut-off points for non-compliance 

In practice, investigators often dichotomise patients to 
be either compliers or non-compliers. A cut-off of 80% 
is commonly used. 9 10 Patients are considered to be 
compliers if they have complied with at least 80% of the 
intervention according to protocol. A cut-off of 100% 
has also been used such that patients are considered to 
be compliers only if they have complied with the entire 
treatment protocol. Compliers are expected to receive 
the full effect of an intervention. We conducted a sensi- 
tivity analysis to investigate the impact of these two 
cut-offs on dichotomising compliers. A new compliance 
indicator for patient i was defined as: tpl(d>0.8) for a 
cut-off of 80% and ti=I(di=l) for a cut-off of 100%. The 
indicator function I returned 1 if the condition was satis- 
fied and 0 otherwise. We then performed the same ana- 
lysis by replacing di with ti for patient i. 



RESULTS 

The estimates by different analyses under the simulated 
scenarios, and their bias, MSE and 95% coverage are 
summarised in tables 1 and 2. For a large treatment 
effect (a mean difference of 30, representing 50% 
improvement over the usual care), the ITT estimate was 
considerably biased. The other estimates were unbiased 
when non-compliant behaviours were random. When 
there were always-takers and never-takers, the PP and 
CACE estimates were the least biased if never-taking 
behaviours were dependent on patients' conditions 
(scenarios C and E). The IV estimate was the least 
biased if non-compliant behaviours were symmetrically 
dependent on patients' conditions (scenarios A and D). 
When there were only never-takers, the PP estimates 
were mostly unbiased. Similar results were found for a 
moderate treatment effect (details not shown) . When an 
intervention was not different from the usual care 
(details not shown), all estimates were unbiased when 
non-compliant behaviours were random. The ITT esti- 
mate was also unbiased if non-compliant behaviours 
were symmetrically dependent on patients' conditions. 
For the remaining scenarios, the ITT estimate was the 
least biased and the other estimates were biased with a 
different degree. 

The IV estimate generally had a larger MSE than the 
other estimates. That was because the standard IV esti- 
mator was sensitive to non-compliant rates. For example, 
when non-compliant rate was equal between groups, the 
denominator was zero and the estimate became 
undefined. When never-takers and always-takers were 
considered, the ITT and IV approaches generally had a 
better 95% coverage than the other approaches. When 
there were only never-takers, the PP approach had the 
best 95% coverage. For a large treatment effect, the ITT 
approach had zero 95% coverage. Overall, the results 
from comparing the MSE and 95% coverage were con- 
sistent with those from comparing the bias of estimates. 

In the sensitivity analysis, we compared the impact of 
using a cut-off of 80% or 100% to dichotomise compli- 
ant patients. The results showed that dichotomising 
patients by a cut-off of 80% resulted in less biased esti- 
mates than dichotomising patients by a cut-off of 100%. 
For a null treatment effect, the treatment estimates 
obtained by applying a cut-off of 80% were less biased 
than those obtained by directly analysing patients on the 
observed degree of compliance. 

DISCUSSION 

Through simulation, we compared different methods of 
analysing non-compliant RCT data. Our results showed 
that the ITT approach was the most optimal when esti- 
mating a null effect since it provided an unbiased or the 
least biased estimate in different scenarios. This result 
was consistent with the general opinion that the ITT esti- 
mate is conservative towards the null. However, for the 
case of a large or moderate treatment effect, the ITT 
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Table 1 Summary of the results when never-takers and always-takers were allowed (treatment effect=30) 



Scenario 



ivieinou 


All or none 








Partial non-compliance 




tsiimaie 


Bias 


IVIOlZ 


coverage \/o) 


Estimate 


Bias 


MSE 


LrO 


ITT 


12 


-18 


330 


0 


10 


-20 


403 


0 


AT 


30 


0 


1 


96 


30 


0 


1 


95 


PP 


30 


0 


1 


95 


30 


0 


2 


96 


IV 


30 


0 


5 


96 


30 


0 


7 


95 


CACE 


30 


0 


1 


95 


30 


0 


2 


95 


ITT 


5 


-25 


651 


0 


4 


-26 


689 


0 


AT 


37 


7 


50 


0 


37 


7 


56 


0 


PP 


36 


6 


38 


0 


37 


7 


50 


0 


IV 


29 


-1 


80 


96 


28 


-2 


129 


96 


CACE 


36 


6 


38 


0 


37 


7 


50 


0 


ITT 


10 


-20 


416 


0 


8 


-22 


483 


0 


AT 


35 


5 


27 


0 


35 


5 


27 


0 


PP 


35 


5 


27 


0 


35 


5 


27 


1 


IV 


35 


5 


34 


60 


35 


5 


38 


66 


CACE 


35 


5 


27 


0 


35 


5 


27 


0 


ITT 


7 


-23 


539 


0 


6 


-24 


590 


0 


AT 


33 


3 


8 


18 


34 


4 


19 


2 


PP 


31 


1 


3 


80 


33 


3 


9 


48 


IV 


25 


-5 


41 


73 


25 


-5 


46 


80 


CACE 


31 


1 


3 


79 


33 


3 


9 


47 


ITT 


5 


-25 


645 


0 


4 


-26 


690 


0 


AT 


23 


-7 


50 


0 


23 


-7 


56 


0 


PP 


24 


-6 


39 


0 


23 


-7 


51 


0 


IV 


31 


1 


174 


97 


34 


4 


4895 


96 


CACE 


24 


-6 


39 


0 


23 


-7 


51 


0 


ITT 


10 


-20 


413 


0 


8 


-22 


483 


0 


AT 


27 


-3 


8 


18 


26 


-4 


19 


4 


PP 


29 


-1 


3 


78 


27 


-3 


9 


50 


IV 


35 


5 


42 


74 


35 


5 


49 


81 


CACE 


29 


-1 


3 


78 


27 


-3 


9 


48 


ITT 


7 


-23 


537 


0 


6 


-24 


591 


0 


AT 


25 


-5 


26 


0 


25 


-5 


27 


0 


PP 


25 


-5 


27 


0 


25 


-5 


27 


0 


IV 


25 


-5 


35 


58 


25 


-5 


39 


66 


CACE 


25 


-5 


27 


0 


25 


-5 


27 


0 



Coverage (%) 



Random 



E 
F 



AT, as treated; 
protocol. 



CACE, complier average casual effect; ITT, intention to treat; IV, instrumental variable; MSE, mean square error; PP, per 



approach was much more biased than the other 
approaches. When patients' non-compliant behaviours 
were purely random, the AT, PP, IV and CACE 
approaches all provided unbiased estimates. For other 
non-compliant scenarios that we considered, the choice 
of optimal method varied. Figure 3 summarises the 
choices of methods under different scenarios to 
produce an unbiased or less biased estimate. 

Although the ITT method is the most commonly 
reported analysis, other analyses of non-compliant data 
may provide a better estimate. Thus, understanding the 
extent of bias for different analyses is important when 
choosing an optimal approach and interpreting the 
results. Our results are limited by a number of factors. 
First, we did not consider specific prognostic factors in 
the simulation. Adjusting for prognostic factors may 
improve the estimation of treatment effect. However, we 



did consider different associations between patients' out- 
comes and non-compliant behaviours. Second, we 
assumed that the clinical effect of an intervention was 
proportional to the degree of compliance. This linear 
association might not represent all real-life situations. 
Third, we did not consider missing data in the simulation 
and assumed that non-compliers' outcomes were still col- 
lectable. Alternatively, imputation techniques can be 
applied to handle missing data. 26 Finally, we only simu- 
lated a subset of general non-compliant scenarios. Thus, 
our findings may not be generalisable to other scenarios. 

Despite the limitations, our study has several strengths. 
The simulation framework was built on three key factors 
of non-compliance: the type of non-compliers, the ran- 
domness of non-compliance and the degree of non- 
compliance. These three factors were not considered 
simultaneously in previous studies. We generated a total 
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Table 2 Summary of the results when only never-takers were allowed (treatment effect=30) 



Scenario 



ivieinou 


All or none 








Partial non-compliance 




tsiimaie 


Bias 


MQC 
IvlOC 


coverage i/oj 


Estimate 


Bias 


MSE 


coverage \/o) 


ITT 


18 


-12 


145 


0 


18 


-12 


145 


0 


AT 


28 


-2 


7 


36 


30 


0 


1 


94 


PP 


28 


-2 


4 


65 


30 


0 


1 


94 


IV 


30 


0 


2 


95 


30 


0 


2 


95 


CACE 


30 


0 


2 


95 


33 


3 


12 


36 


ITT 


10 


-20 


389 


0 


10 


-20 


387 


0 


AT 


30 


0 


1 


94 


32 


2 


5 


60 


PP 


29 


-1 


3 


76 


30 


0 


2 


96 


IV 


25 


-5 


33 


35 


25 


-5 


31 


38 


CACE 


26 


-4 


15 


34 


30 


0 


3 


94 


ITT 


15 


-15 


239 


0 


15 


-15 


241 


0 


AT 


26 


-4 


16 


9 


28 


-2 


5 


63 


PP 


28 


-2 


6 


61 


30 


0 


2 


94 


IV 


35 


5 


32 


41 


35 


5 


31 


39 


CACE 


34 


4 


16 


32 


36 


6 


4 


8 



Random 



AT, as treated; CACE, complier average casual effect; ITT, intention to treat; IV, instrumental variable; MSE, mean square error; PP, per 
protocol. 



of 60 scenarios by varying non-compliant factors and the 
magnitude of treatment effect (ie, null, moderate or 
large). The findings will help investigators choose the 
optimal approaches when dealing with similar non- 
compliant problems. Our results also confirm some 



previous findings. For example, the ITT analysis was 
unbiased if the treatment effect was zero. 9 All estimates 
were unbiased if non-compliance was independent of 
patients' outcomes and the IV estimate was also unbiased 
when non-compliance was symmetrically dependent on 



Expected effect of the intervention relative to the 
control 



A moderate or larger effect: 25% or 
more improvement to the control 



A null to small effect: 0% to less than 
25% improvement to the control 



Random non-compliance 



Dependent non-compliance 



The ITT analysis 



The AT, PP, IV, 
and CACE analyses 



Only never-takers Always- and never-takers 



Scenarios C and E 



The PP 
analysis 



Scenarios A and D 











The PPand 
CACE analyses 




The IV 
analysis 



Figure 3 Choosing optimal analyses for different non-compliant scenarios. ITT, intention to treat; AT, as treated; PP, per 
protocol; IV, instrumental variable; CACE, complier average causal effect. Scenario A: patients with good conditions will always 
get the intervention and those with poor conditions will always reject it. Scenario C: patients with poor conditions will always 
reject the intervention. Scenario D: patients with good conditions will always reject the intervention and those with poor conditions 
will always get it. Scenario E: patients with good conditions will always reject the intervention. A good condition was defined to 
have an outcome score at least 0.5 SDs above the group mean under usual care. A poor condition was defined to have an 
outcome score at last 0.5 SDs below the group mean under usual care. In addition, it was assumed that the intervention and 
usual care were the only treatment option. 
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patients' outcomes. 17 In addition, we found that the PP 
estimate was unbiased when there were only never- 
takers. While the real impact of non-compliance on esti- 
mating treatment effect is difficult to generalise, we have 
compared the performance of common analyses under 
specific non-compliant scenarios. The results highlight 
the value of employing multiple approaches to analyse 
non-compliant data. Our work has considered additional 
non-compliant scenarios that were not considered by 
previous studies. It also contributes to the quality assess- 
ment of research evidence generated from RCTs subject 
to non-compliance and provides basis for a more 
complex evaluation. 

CONCLUSION 

Our simulation shows that the ITT analysis under non- 
compliance is considerably biased when an intervention 
has a large effect over the control. Alternative analyses 
can provide unbiased or less biased estimates. For RCTs 
subject to non-compliance, we make some suggestions 
for the choice of analyses under specific scenarios to 
minimise the bias of estimated treatment effect. Our 
study also informs the design of further investigations on 
the issue of non-compliance in RCTs. 
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